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Background Art 

Advances in a solid state detector technology lead to an increasing use of focal plane 
array (FPA) detectors for infrared (IR) and visible-light imaging and spectral sensing. Use of 
FPAs allows compactness, cost-effective production and high performance of sensors, in turn, 
5 leading to high sensitivity and resolution. The IR-sensor arrays are applicable to broadband 
thermal imaging, where maximizing broadband spatial resolution and intensity signal-to-noise 
ratio is of paramount importance. Some of the examples of use of such FPAs include night- 
vision systems, airborne and space-based reconnaissance and surveillance systems, astronomical 
imaging, and forest fire early detection systems. 

10 Other examples of use of such array sensors include spectral sensing. In spectral sensing, 

different array elements are designed to respond to different wavelengths of light. This allows 
the sensor to detect the spectrum of the source. Some of the modern day spectral sensors are 
capable of measuring irradiance at upwards of 200 narrow bands in every point in the image. 
Some examples of spectral sensors include detection and identification systems for hazardous 

15 biological and chemical agents, as well as monitoring systems of environmental changes in lakes 
and other natural habitats. 

However, current technology relating to array-sensors have several drawbacks and 
disadvantages. One major disadvantage to using modern day array-sensors for quantitative 
remote sensing is the array's non-uniformity noise. Non-uniformity noise primarily results from 

20 minute detector-to-detector dissimilarities due to FPA's pixel-to-pixel variation in the response 

as well as fabrication related geometrical dissimilarities in individual FPA elements. It is well 

known, that most modern FPAs suffer from spatial non-uniformity noise. Spatial non-uniformity 

noise manifests itself in a form of a quasi-fixed pattern, also referred to as fixed-pattern noise 

("FPN"). Non-uniformity noise can lead to an inaccurate radiometric measurement, reduced 
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temperature resolvability, and reduced spatial resolution. Furthermore, it has a detrimental effect 
on the performance of many post-processing enhancement and restoration algorithms that 
assume temporal noise but not FPN, whose temporal statistics exhibit a high degree of 
correlation. 

5 Some modern-day array-sensors developed systems for compensation of the non- 

uniformity noise through an one-time factory calibration of the sensor. However, such systems 
do not compensate for sensor drift problems over time, thus, making the one time factory 
calibration ineffective. To correct for sensor drift over time, a constant calibration of the sensor 
is necessary to correct for non-uniformity in the detector response. However, present day 

10 calibration techniques disrupt operation of the array sensor during the calibration process. 
Furthermore, there are other problems associated with modern day FPA sensors. 

Therefore, there is a need for a better focal plane array sensor capable of being 
continuously calibrated while still detecting images from the scene. Furthermore, there is a need 
for a focal plane array sensor capable of continuously correcting for non-uniformity drift. 

15 Further features and advantages of the invention, as well as the structure and operation of 

various embodiments of the invention, are described in detail below with reference to the 
accompanying drawings. It is noted that the invention is not limited to the specific embodiment 
described herein. Such embodiments are presented herein for illustrative purposes only. 
Additional embodiments will be apparent to persons skilled in the relevant art(s) based on the 

20 teachings contained herein. 
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BRIEF SUMMARY OF THE INVENTION 



The present invention relates to a field of optical imaging and spectral sensing. In 
particular, the present invention relates to a non-disruptive continuous and radiometrically 
5 accurate calibration of a focal plane array sensor using global scene motion. 

In an embodiment, the present invention is a method for generating a calibrated image 
sequence of a scene using a focal plane array detector having a plurality of outer perimeter 
detectors, a plurality of inner detectors and a plurality of buffer detectors placed adjacent to the 
plurality of outer perimeter detectors and the plurality of inner detectors. To generate the 
10 calibrated image sequence of the scene, the method first redirects the field-of-view of the 

plurality of outer perimeter detectors away from the scene and towards a calibration source using 
an optical switch. Then, it calibrates outer perimeter detectors using the calibration source and 
again redirects the field-of-view of the plurality of outer perimeter detectors away from the 
calibration source and towards the scene using an optical switch. After calibrating the outer 
15 perimeter detectors, the non-uniformity correction ("NUC") algorithm collects a sequence of 
images of the scene for a particular period of time using the plurality of outer perimeter 
detectors, buffer detectors, and inner detectors. Readout signals for each detector in the plurality 
of outer perimeter detectors, buffer detectors, and inner detectors are generated, where the 
readout signals are expressed as follows: 
20 y n (*. j) = a n,s 0\ J)z n (/, j) + K (i, j) 

where a n ,s(i, j) and b n (i, j) are gain and bias associated with an ij th detector at time n, 
respectively; z n (i, j) is a total average number of photons impinging on the ij* detector at time n. 
Using the readout signals, the NUC algorithm computes bias correction for each detector 

in the focal plane array. Then, it generates a corrected image sequence using the collected 
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sequence of images of the scene and bias correction. The NUC algorithm is repetitive, because 
calibration of detectors and collection of images is performed continuously throughout operation 
of the NUC algorithm. 

Another embodiment of the present invention relates to an optical apparatus, having an 
5 imaging mechanism for generating a corrected image sequence of a scene using non-uniformity 
correction techniques described above. The optical apparatus includes a first off-axis parabolic 
mirror, a second off-axis parabolic mirror and a movable field stop, where the movable field stop 
is placed in an optical path between the first and the second off-axis parabolic mirrors. A light 
incident on the first off-axis parabolic mirror is reflected towards the second off-axis parabolic 

10 mirror via the movable field stop. The movable field stop acts as an optical switch and in this 
embodiment also acts as the calibration source. Light passing through the field stop travels 
towards the second off-axis parabolic mirror, which reflects it towards the imaging mechanism. 

Yet another embodiment of the present invention relates to a method of generating an 
image sequence that includes detecting scene irradiance using detectors in a focal plane array, 

15 generating an output image sequence for each of the detectors based on the detected irradiance, 
and correcting the output image sequence generated by a first subset of detectors in the focal 
plane array and the output image sequence generated by a second subset of detectors in the focal 
plane array using the correction provided to the first subset of detectors. 

Further features and advantages of the invention, as well as structure and operation of 

20 various embodiments of the invention, are disclosed in detail below will reference to the 
accompanying drawings. 



#81735 vl 
103752-48754 



BRIEF DESCRIPTION OF THE FIGURES 



The present invention is described with reference to the accompanying drawings. In the 
drawings, like reference numbers indicate identical or functionally similar elements. 
5 Additionally, the left-most digit(s) of a reference number identifies the drawing in which the 
reference number first appears. 

FIG. 1 illustrates a non-uniformity correction ("NUC") system showing a focal plane 
array sensor, according to the present invention. 

FIG. 2a illustrates an assembly of detectors in the focal plane array sensor before 
10 application of a non-uniformity correction ("NUC") algorithm, according to the present 
invention. 

FIG. 2b illustrates focal plane array sensor's detectors of FIG. 2a during the application 
of the NUC algorithm, according to the present invention. 

FIG. 2c illustrates a direction of the NUC algorithm's application to the focal plane 
15 array's detectors, according to the present invention. 

FIG. 3 is a flow chart illustrating steps of the NUC algorithm, according to the present 
invention. 

FIG. 4 illustrates an optical system setting that employs the NUC algorithm, according to 
the present invention. 

20 FIG. 5a illustrates an infrared image of a scene generated using a conventional. 

FIG. 5b illustrates a corrected infrared image sequence of the scene in FIG. 5a using a 
conventional two-point calibration technique. 

FIG. 5c illustrates a corrected infrared image sequence of the scene in FIG. 5a using the 
present invention's NUC algorithm. 
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FIG. 5d illustrates a corrected infrared image sequence of the scene in FIG. 5a employing 
present invention's generalized NUC algorithm. 
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DETAILED DESCRIPTION OF THE INVENTION 
Table of Contents. 

1. Overview. 

2. Focal Plane Array. 

5 3. A Non-Uniformity Correction Algorithm for Generating a Corrected Image of a 

Scene. 

4. Correction for non-uniform gain. 

5. Optical System. 

6. Conclusion. 

10 

While the present invention is described herein with reference to illustrative 
embodiments for particular applications, it should be understood that the invention is not limited 
thereto. Those skilled in the art with access to the teachings provided herein will recognize 
additional modifications, applications, and embodiments within the scope thereof and additional 
15 fields in which the present invention would be of significant utility. 

1. Overview. 

In an embodiment, the present invention brings radiometric accuracy into a non- 
disruptive scene-based non-uniformity correction ("NUC") by continuously calibrating perimeter 
20 detectors within a focal plane array detector ("FPA"). It also relies on a particular image- 
sequence processing NUC algorithm to calibrate inner detectors of the FPA. 

In an alternate embodiment, the present invention provides an optical system which 
allows for calibration of the perimeter detectors and generation of a corrected image. 
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Another embodiment of the present invention provides non-uniformity correction using a 
generalized bias-independent approach to correcting FPA detectors' generated scene images that 
does not require perimeter calibration. 



5 2. Focal Plane Array. 

FIG. 1 illustrates an embodiment of a non-uniformity correction ("NUC") system 100, 
according to the present invention. The NUC system 100 includes a scene 104, a focal plane 
array ("FPA") 106, a calibration source 102 and an optical switch 120. The FPA 106 is in an 
optical path between the calibration source 102 via the optical switch 120. The optical switch 
10 120 is in an optical path between the FPA 106 and the calibration source 120. 

FPA 106 includes a plurality of detectors 115 (a, b, ... , mo), (referred to as detectors or 
pixels 115). Each detector 115 of FPA 106 is a light sensing element. Upon detection of 
irradiance from the scene 104, a detector 115 generates a readout signal in a form of a voltage 
associated with that irradiance. The strength of the readout signal depends on the level of 
15 irradiance received. For a particular detector 1 15, the detector's readout signal can be expressed 
as follows: 

y n 0'. J) = a n,s & ;X (». J) + K ;) (!) 

where y n (i, j) is the readout signal for a particular if* detector 1 15 at time n; a njS (i, j) and 
b n (i, j) are the gain and bias associated with the ij th detector at time n, respectively. The gain and 
20 bias depend on the frame time n and capture any possible temporal drift in their values. Further, 
the gain depends on the spectrum S of the irradiance source. The quantity z n (i, j) is the total 
average number of photons impinging on the ij th detector (integrated over the wavelengths 
supported by the detector) at time n. Further, the term a n ,s(i, j) collectively represents the 
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detector quantum efficiency over the spectrum S, the optical transmission (from the source to the 
if 1 detector), the detector's active area and other optical and sensor-specific factors. The term 
b n (i, j) represents the detector's bias and it also includes the contribution due to dark current. As 
seen from equation (1), gain's and bias's dependence on detector's position gives rise to non- 
5 uniformity noise. Furthermore, the detector's wavelength-dependent quantum efficiency, which 
is included in the gain term, gives the non-uniformity noise a spectral dependence, as well. 

Detectors 115 include outer perimeter detectors 112(a, b, ... , mi) (referred to as outer 
perimeter detectors 112), secondary perimeter or buffer detectors 1 14(a, b, . . . , m 2 ) (referred to as 
secondary perimeter or buffer detectors 1 14), and inner detectors 1 16(a, b, . . . , m3) (referred to as 

10 inner detectors 1 16). In the FIG. 1 embodiment, FPA 106 has a rectangular shape with detectors 
115 forming rows (1, 2, ... i ...p) and columns (1, 2, ... j ...1) . Therefore, each detector 115 is 
identified by a coordinate pair (i, j). Alternate embodiments of FPA 106 can include various 
shapes and sizes of FPA 106, which are application specific. 

As shown in FIG. 1, outer perimeter detectors 1 12 occupy an outer edge of FPA 106. 

15 The outer perimeter detectors 112 form one or more rows along the outer edge of FPA 106. The 
buffer detectors 1 14 are parallel to the outer edge of FPA 106 and adjacent to the outer perimeter 
detectors 112 and the inner detectors 1 16. The buffer detectors 1 14 form at least one row in the 
FPA 106. The inner detectors 116 are part of the inner portion of FPA 106. Because each 
detector 1 15 is identified by the coordinate pair (i, j), each outer perimeter detector 1 12, buffer 

20 detector 1 14 and inner detector 1 16 is also identified by coordinate pair (i, j). As can be 

understood by one having ordinary skill in the relevant art, other arrangements of outer perimeter 
detectors 1 12, buffer detectors 1 14 and inner detectors 1 16 are possible. Also, the locations of 
the detectors are not limited to the FIG. 1 embodiment. 
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Calibration source 102 is optically coupled to FPA 106 via optical switch 120. 
Calibration source 102 serves as a source of calibration information for FPA 106. In an 
embodiment, calibration source 102 can be a temperature controlled blackbody source defined 
by the blackbody energy density function. In an alternate embodiment, calibration source 102 
5 can be a two-point calibration system. As can be understood by one having ordinary skill in the 
relevant art, other calibration sources can be used depending on the desired accuracy of 
calibration. 

Optical switch 120 optically separates calibration source 102 and FPA 106. In particular, 
optical switch 120 optically separates the calibration source 102 and the outer perimeter 

10 detectors 1 12 of FPA 106. Calibration source 102 supplies calibration information 125 to the 
outer perimeter detectors 112, when optical switch 120 is in an "on" position 121. This means 
that the field of view (FOV) of the outer perimeter detectors 1 12 is unobstructed towards the 
calibration source 102 but is blocked by the optical switch towards scene 104. Calibration 
information 125 is not supplied to the outer perimeter detectors 112 when calibration source 102 

15 is blocked and optical switch 120 is in an "off position 122. This means that the field of view 
of the outer perimeter detectors 1 12 is blocked by the optical switch 120 towards calibration 
source 102 and is unobstructed towards scene 104. In an embodiment, the optical switch 120 
and the calibration source 102 can be incorporated into a single temperature controlled movable 
field stop. As can be understood by one having ordinary skill in the relevant art, optical switch 

20 120 can be any optical element capable of changing the field of view of the outer perimeter 
detectors 112. 

Scene 104 is any object that is desired to be imaged by the FPA 106. FPA 106 measures 
irradiance 130 directed from scene 104. Scene 104 can have various spectral and spatial 
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irradiance patterns. For example, scene 104 can be sky, water, camouflage, or any other thermal 
media. 

In this embodiment, NUC system 100 operates simultaneously in two cycles: the 
calibration cycle and the scene cycle. During the scene cycle, all of the detectors 115 have their 
5 field of view (FOV) directed towards scene 104. Therefore, each detector 115 generates a 

response signal based on an irradiance 130. The optical switch 120 is in the "off position 122, 
which allows outer perimeter detectors 1 12 to view the scene 104. 

During the calibration cycle, the optical switch 120 is in the "on" position 121 directing 
the field of view of the outer perimeter detectors 1 12 to the calibration source 102. The buffer 

10 detectors 1 14 and the inner detectors 116 have their field of view directed towards the scene 104 
at all times. During the cycle, the calibration source 102 transfers calibration information to the 
outer perimeter detectors 1 12 to calibrate the detectors 1 12. Thus, the buffer detectors 1 14 and 
the inner detectors 112 are never blocked by the optical switch 120 during the calibration cycle 
or the scene cycle. The calibration source 102 transfers calibration information to outer 

15 perimeter detectors 112 using a conventional calibration technique, such as a blackbody source, 
or a two-temperature calibration system, or any other calibration system. 

After outer perimeter detectors 112 are calibrated by the calibration source 102, the field 
of view of the outer perimeter detectors 1 12 is re-directed to the scene 104 by optical switch 120 
and calibration information is transferred to buffer detectors 114 without interrupting the fields 

20 of view of the buffer detectors 1 14 or the inner detectors 1 16. Furthermore, while the calibration 

information is being transferred to buffer detectors 1 14, the calibration information is also being 

transferred to inner detectors 1 16 by buffer detectors 1 14. This is done without interrupting the 

fields of view of the inner detectors 1 16 or the buffer detectors 1 14. A non-uniformity correction 

algorithm, described with reference to FIGS. 2a-c and 3, allows transfer of calibration 
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information among the FPA detectors. While the outer perimeter detectors 112 are being 
calibrated, the buffer detectors 14 continue to transfer the calibration information from the 
previous calibration cycle to the inner detectors 116. 



5 3. A Non-Uniformity Correction Algorithm for Generating a Corrected Image of a 

Scene. 

FIGS. 2a-2c illustrate transfer of calibration information from calibrated detectors 115 to 
uncalibrated detectors 115 using the NUC algorithm. Referring to FIG. 2a, a portion of FPA 106 
having four adjacent detectors 1 15(a, b, c, d) is shown. Detector 1 15a is identified by the 

10 coordinate pair (i, j). Detector 1 15b is identified by the coordinate pair (i, j-1). The other 

detectors are similarly identified using respective coordinate pairs, as shown in FIG. 2a. For the 
purposes of the following description, it is assumed that the detectors 115b, 115c, and 115d have 
already been calibrated. These detectors can be either outer perimeter detectors 112, buffer 
detectors 1 14, or inner detectors 1 16 that were calibrated either using the calibration source 102 

15 (calibrating outer perimeter detectors 1 12) or the NUC algorithm (calibrating buffer detectors 
1 14 and inner detectors 1 16). Using the NUC algorithm, the calibration information is 
transferred from calibrated detectors 115b, 115c, and 115d to uncalibrated detector 115a, thereby 
calibrating detector 1 15a, as shown in FIGS. 2a-b. During calibration process, the NUC 
algorithm uses detector readout information from all four detectors 1 15(a, b, c, d) to determine 

20 calibration needed for detector 1 15a. 

FIG. 2a shows four detectors 115(a, b, c, d) at time (k-1) before transfer of calibration 

information from the calibrated detectors 115b, 115c, and 115d to the uncalibrated detector 115a. 

Calibrated detectors 115b, 115c, and 115d are identified by coordinate pairs (i, j-1), (i-1, j-1), 

and (i-1, j), respectively, and are shown in gray in FIG. 2a. 
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Referring to equation (1) above, because detectors 115b, 115c, and 115d were previously 
calibrated, their gains are equal to one (a n , s (i, j) = 1) and their biases are equal to zero (b n (i, j) = 
0). The bias of the uncalibrated detector 1 15a is not equal to zero. Upon application of the NUC 
algorithm, the bias of detector 1 15a is adjusted to zero. In this embodiment, the gain of the 
5 uncalibrated detector 115a is equal to one. However, the present invention's NUC algorithm 
calibrates detector 1 15a even if its gain is not uniform. In an alternate embodiment described 
below, the present invention's NUC algorithm applies non-uniformity correction to detectors 
using an initial arbitrary unknown bias value, as opposed to initial zero bias value. The initial 
arbitrary bias value is independent of any parameters of the algorithm and the system employing 

10 the algorithm. There is no need to have any knowledge or control over this value. This allows 
application of the non-uniformity correction using the NUC algorithm without knowledge of 
initial bias values or any calibration parameters. 

FIG. 2b shows transfer of the calibration information from calibrated detectors 115b, 
1 15c, and 1 15d to the uncalibrated detector 1 15a. During the transfer of calibration information 

15 at time k, the field of view of the uncalibrated detector 1 15a will "straddle" the field of view of 
the four detectors 115(a, b, c, d) at time (k-1), due to scene motion. This is shown by gray 
portions yi, y 2 , Y3> and y 4 corresponding to detectors 1 15(a, b, c, d), respectively. In other words, 
at time k, the calibration information from calibrated detectors 115b, 115c, and 115d along with 
readout signals from detectors 1 15a, 1 15b, 1 15c, and 1 15d at times k and (k-1) is used to 

20 compute non-uniformity correction for detector 1 15a (described below). After completion of 
transfer of calibration information, all four detectors 115(a, b, c, d) are calibrated and can 
generate a uniform response to scene irradiance. The calibration process then proceeds to 
adjacent detectors 1 15 in the same manner as described above. This is further illustrated by 
FIG. 2c below. 

14 

#81735 vl 
103752-48754 



Referring to FIG. 2c, the present invention is capable of transferring calibration 
information received from the calibration source 102 in various directions between detectors 115 
within EPA 106 (for example, the rightward and downward directions). The calibration 
information is transferred from calibrated detectors 115 (shown in gray color) down to 
5 uncalibrated detectors 115 (shown in white color) through calibration of a particular detector 
group Gi 115 and then transferring calibration information either in the downward or rightward 
manner to neighboring detectors 115 within detector group Gj. This, in turn, allows calibration 
of detector groups G2, G3, and other detectors 115 of FPA 106. However, as can be understood 
by one having ordinary skill in the relevant art, transfer of calibration information is possible in 

10 other directions at the same time, such as upward and leftward directions. 

FIG. 3 illustrates an embodiment of a NUC algorithm 300 for generating a non- 
uniformity correction to detectors 115 of FPA 106. The NUC algorithm 300 continuously 
calibrates detectors 1 15 to correct for non-uniformities in detectors' 115 responses to the 
irradiance 130. In the FIG. 3 embodiment, the NUC algorithm 300 simultaneously operates in 

15 two cycles: a calibration cycle 315 and a scene cycle 325, as shown in FIG. 3. As mentioned 

above, the NUC algorithm 300 continuously calibrates detectors 115 during calibration cycle 315 
while applying calibration information to correct any non-uniformities in image sequences 
generated during scene cycle 325. 

In an embodiment, the NUC algorithm 300 performs under an assumption that the gain 

20 term of a detector readout signal is one (i.e., a n ,s(i, j) = 1). This means that the gain is uniform 
throughout the calibration and scene viewing cycles. However, the NUC algorithm 300 
substantially eliminates non-uniformities from a detector received image even when the gain is 
not uniform. 
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Referring back to FIG. 3, in step 301 of the calibration cycle 315, the NUC system 100 
(shown in FIG. 1) redirects the field of view of the outer perimeter detectors 1 12 towards the 
calibration source 102. This is accomplished by switching the optical switch 120 into the "on" 
position 121, as described above with reference to FIG. 1. With switch 120 in the "on" position, 
5 the calibration source calibrates the outer perimeter detectors 1 12. At the same time, the buffer 
detectors 1 14 and the inner detectors 1 16 simultaneously detect the scene irradiance 130 from 
scene 104. During this interval, the buffer detectors 114 transfer calibration information, 
received from an earlier calibration cycle, to the inner detectors 116. 

In step 302, the outer perimeter detectors 1 12 are calibrated using calibration information 

10 transferred to outer perimeter detectors 112 from calibration source 102. During the calibration 
process, the bias term, b D (i, j), of the outer perimeter detectors 1 12 is equal to zero. Using the 
calibration information received by the outer perimeter detectors 1 12, the algorithm computes a 
bias correction matrix A(i, j). The bias correction matrix A(i, j) determines the non-uniformity 
correction for a particular detector 115 designated by (i, j). 

15 Once the calibration of the outer perimeter detectors 1 12 is completed, the NUC system 

100 redirects the field of view of outer perimeter detectors 1 12 away from the calibration source 
102. This is accomplished by changing the optical switch 120 to an "off position 122 and 
allowing the outer perimeter detectors 1 12 to view scene 104. This is shown in step 303 of the 
NUC algorithm 300. 

20 The NUC algorithm 300 then begins scene cycle 325 by proceeding to step 304. Once 

the calibration information is transferred to outer perimeter detectors 112, the FPA 106 collects 
sequences of images using an arbitrary inter-frame motion. As would be understood by one 
having ordinary skill in the relevant art, other methods of collecting images from the scene using 
an FPA are possible. 
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For each pair of collected image frames, the FPA 106 estimates a frame-to-frame shift 
using conventional frame shift estimation techniques, as shown in step 305. The NUC algorithm 
300, described by equations (2) - (10) below, applies to each pair of frames to correct for any 
non-uniformities associated with the image detection. 
5 The processing then proceeds to steps 306a and 306b. In steps 306a and 306b, detectors 

generate readout signals y n (i, j), according to equation (1) above (where n = 1, k, where k is 
an integer). In step 306a, a readout signal y k (i, j) is generated for all i and j at time k. In step 
306b, a readout signal y k+ i(i, j) is generated for i and j at time k+1. The following equation 
represents the readout signal y k (i, j) for the ij th detector at time k: 
10 y k (iJ) = Z k (iJ) + b(iJ) (2) 

Assuming gain uniformity and a vertical calibration movement by a shift amount a 
(amount determined in step 305 as a frame-to-frame shift). The vertical calibration movement is 
a direction of calibration that starts from the top FPA detectors and then proceeds to the bottom 
FPA detectors - a one-dimensional motion. The following equation represents readout signal 
15 y k+ i(i+l, j) for the (i+l)j th detector at time (k+1): 

y M (i + 1, j) = az k (i, ;) + (1 - a)z k (i + 1, ;) + b(i + 1, j) (3) 
Using an one dimensional motion, the bias correction matrix A(i, j) is calculated in step 
307 using the following equation: 

A(i + 1, j) = y k (i, j) - (1 - a x )y k (i + 1, ;') - y M (i + 1, j) (4) 
20 which after simplification and canceling out of terms, equation (4) becomes: 

Mi + lJ) = b(iJ)-b(i + lJ) (5) 
The bias correction matrix A(i+1, j) is computed using the readout signals for detectors 
that are immediately adjacent to the detector for which the bias correction is being calculated. 
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Further, the bias correction matrix A(i, j) is sequentially calculated for each detector in each 
column of the FPA starting with a first detector (for example, (2, j)) and moving downward. The 
bias correction matrix A(i, j) is computed for each pair of frames. The values of A(i, j) for each 
pair of frames can be combined to provide a more robust correction matrix. This combination 
5 can be performed, for example, by averaging the matrices for each pair of frames, though other 
methods are possible. Thus, the bias correction matrix is continuously updated using equation 
(5) as the NUC algorithm 300 proceeds from frame to frame. If calibration information (or 
readout signals) are known for ij lh and (i+l)j th detectors at time k, the bias correction can be 
computed for (i+l)j th detector at time k+1 and beyond. 
10 For example, if i=l and j=l, then 

y M (2,1) = az k (1,1) + (1 - a)z k (2,1) + 6(2,1) (6) 
and 

y 4 (U) = z t (U) + *(U) (7) 
then the bias correction matrix A(2, 1) is computed using equations (3)-(5) as follows: 
15 A(2,l) = y k (1,1) - (1 - a x )y k (2,1) - y M (2,1) = 6(1,1) - 6(2,1) (8) 

Therefore, if A(2, 1) is added to the (2, l) th detector of any raw frame, the bias of this 
detector will be converted from b(2, 1) to b(l, 1). However, since the bias of detector (1, 1) is 
equal to zero (detector (1, 1) was calibrated in step 301 using a calibration source and, thus, its 
bias is zero), the bias of the (1, 2) th detector is now equal to zero, because of application of non- 
20 uniformity correction. 

The one-dimensional calibration process is used when a top row of FPA detectors is 
already calibrated by the calibration source 102. However, multiple pairs of detectors at 
different times can be utilized and the same procedure can be repeated for each detector at each 
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particular time. The following is a description of the utilization of the NUC algorithm 300 for an 

arbitrary application of non-uniformity correction to selected detectors 115 at various times k, 

without limiting such application to a one-dimensional motion. 

Assuming that a and p denote arbitrary vertical and horizontal directional frame shifts 

5 between detectors' responses to scene irradiance, respectively, between and (k+l) th time 

frames, as determined by step 305 of the NUC algorithm 300, then each shift is written as its 

whole-integer (denoted by "[ ]") plus its fractional part (denoted by "{ }"), i.e., a = [a] +{a} and 

P = [PI +{P1- Using the linear interpolation model described by equation (1) above, the bias 

correction matrix A is calculated for the (i, j) th detector at time (k+1) using values of its 

10 neighboring detectors (as shown in FIGS. 2a-2c): 

A(/, ;) = {a} {J3}y k (i - [a] -1J- [0] - 1) + (1 - {a}){0}y k (i - [a], j - [J3] - 1) 

+ {a}(l - {J3})y k (i - [a] -IJ- [0]) (9) 
+ (1 - {a)YX -{J3})y k (i - [a], j - [J3]) - y k (i, j) 

After substituting appropriate values for y k (i, j) from equation (1), the following is a 
result for the bias correction matrix A(i, j): 

A(/, j) = {a} {/3}b{i - [a] -1J- [0] - 1) 

(10) 

+ (1 -{a})(l - {J3})b(i - [a], j - [0]) - b{U j) 

15 Therefore, the top-most and left-most layers of detectors of the FPA 106 are already 

calibrated [a] + 1 and [P] + 1 detectors "deep," respectively. All of the respective biases in 

equation (10) are forced to zero except for b(i, j). The bias correction matrix A(i, j) = -b(i, j), 

which is the bias correction for the (i, j) th detector. 

The NUC algorithm 300 calculates the bias correction matrix A(i, j) by proceeding in 

20 downward and rightward directions among the detectors in the focal plane array for each pair of 

frames. Once the bias correction matrix is computed for each pair of frames, the NUC algorithm 
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300 combines this correction matrix with those obtained from previous pairs of frames to 
compute a more robust correction matrix for the focal plane array. The global A can be 
computed using conventional methods, such as averaging all A(i, j)'s or other methods. The 
global A serves as a correction to all of the detectors in the focal plane array. 

Referring back to FIG. 3, after computing the bias correction matrix A(i, j), the NUC 
algorithm applies the bias correction matrix A(i, j) to the sequence of images collected in step 
304. As described above, the bias correction matrix A(i, j) corrects the bias for each detector 1 15 
of FPA 106 being calibrated and, thus, corrects any non-uniformities associated with a detector 
response to scene irradiance 130 generated by the FPA 106. 

After each detector that had its field of view directed towards scene 104 is corrected 
using the bias correction matrix A(i, j), the NUC algorithm 300 generates a corrected image 
sequence in step 309. The corrected image sequence is a radiometrically accurate image of the 
scene 104. The accuracy of the image depends on a selected accuracy of the calibration source 
102. 

The processing then proceeds to step 310. In step 310, the NUC algorithm 300 
determines whether the calibrated image sequence generated in step 309 is adequate. This step is 
performed through assessment of temporal drift within the detectors. If the calibrated image 
sequence is satisfactory, the processing proceeds to step 304 to gather new images of the scene 
104. 

However, if the calibrated image sequence is not satisfactory, further non-uniformity 

correction is desired. The NUC algorithm then proceeds back to step 301 to perform calibration 

of the outer perimeter detectors 112 and generate new calibrated image sequences. While the 

calibration cycle (including steps 301 through 303) is being performed, the buffer detectors 114 

and inner detectors 116 continue to perform the scene cycle (steps 304 through 309). 
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Because NUC algorithm 300 continuously calibrates detectors 1 15 of FPA 106, steps 301 
through 303 are repeated throughout the operation of the NUC algorithm 300. Moreover, the 
outer perimeter detectors 112 continuously receive calibration information from calibration 
source 102. 

5 The above non-uniformity correction algorithm allows for calibration and imaging of an 

object using a model for an arbitrary two-dimensional motion. The algorithm also applies non- 
uniformity correction in circumstances where bias is equal to some arbitrary value b and without 
initial calibration of the FPA detectors' biases to zero. In this embodiment, the NUC algorithm 
300 substantially involves a two-step approach to generating a corrected image. First step is 

10 directed to unifying bias values of the outer perimeter detectors to a single arbitrary and 

unknown bias value b. Second step is directed to applying non-uniformity correction techniques 
described above to other detectors using the above unknown bias value b. As stated above, the 
NUC algorithm 300 will apply non-uniformity correction regardless of the unknown value b. 
This method is advantageous for systems where absolute radiometry is not necessary and is 

15 applicable to all systems where a video sequence is available with global translational motion. 

The NUC algorithm 300 unifies the bias values of all FPA detectors to an arbitrary bias 
value b of a chosen outer perimeter detector. Therefore, NUC algorithm applies to the outer 
perimeter FPA detectors to achieve an equal, arbitrary bias value b for these detectors. The 
perimeter elements are unified to the same unknown bias value b using techniques such as the 

20 one described in "An algebraic algorithm for nonuniformity correction in focal-plane arrays," by 

Ratliff, B.M., Hayat, M.M., Hardie, R.C., Journal of Optical Society of America, Vol. 19, No. 9, 

September 2002, pg. 1737-1747. Once all outer perimeter detectors have an equal bias value b, 

NUC algorithm 300 applies a two-dimensional model to equate bias values of all FPA detectors 

to the same value b of the outer perimeter detectors, as described above. This method 
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substantially eliminates costly application of perimeter calibration when radiometric accuracy is 
of lesser concern. 

Assuming that all outer perimeter detectors have bias values of b, the if 1 detector's bias 
differential for an image pair with down-right ward two-dimensional motion is computed as 
5 follows (similarly to equations (9)-(10)): 

+ r 4 y k (i - [a] , j - [0]) + y M (i, y) 

where y denotes detector outputs, i = 2+[a], 3+[a], M, j = 2+[p], 3+[P], N, M and 
N denote the size of an image. The vertical and horizontal shift components are denoted by a 
and p, respectively. These shifts can be written as an integer and fractional parts (as described 
10 above with respect to equations (9) and (10)): a = [a] + {a} and p = [p] + {P}, where [] denotes 
an integer part and { } denotes a fractional part. The yi are fractional pixel factors that are defined 
as follows: 

Ti = |{aHP>|; 

Y2=(l-Ua}|)UP}|; 
15 Y3=|{a}|(H{p}|); 

y 4 =(l-|{a}|)(l-|{p}|); 
where yi + 72 + 73 + 74 = 1- Because the outer perimeter detectors have been unified to 
have an unknown bias value of b, the top-leftmost non-unified detector (i.e., detector's bias is not 
equal to b) has a bias differential of the form (using equations (9)-(l 1)): 
20 A(2 + [flr] f 2 + [/?]) = Yxb + V 2 b + 7 3 b + y A b - b(2 + [a] 9 2 + [J3]) (12) 

because the sum of the terms 71 is equal to 1, equation (12) becomes: 
A(2 + [all + [/?]) = b - b(2 + [a],2 + [J3]) (13) 
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Because this bias differential is added to the top-leftmost non-unified detector, y(2+[a], 
2+[p]), its bias is subtracted and replaced with the bias value of b. This equates this detector's 
bias to the outer perimeter detectors' biases. Because this algorithm is applied recursively, the 
buffer detectors' 114 and the interior detectors' 116 bias differentials are computed as follows: 
5 WJ) = b-b(iJ) (14) 

After calculating the updated bias differentials for the FPA detectors, the bias values are 
arranged in a matrix form and are added to an arbitrary raw image for correction. Therefore, the 
generalization of NUC algorithm 300 accommodates large number of detectors. Further, use of 
the arbitrary bias value b allows for applying non-uniformity correction algorithms without use 
10 of the perimeter calibration, where all bias values of the outer perimeter detectors are calibrated 
to zero. Fig. 5d illustrates an image corrected by the above generalized algorithm. 

In an alternative embodiment of the present invention, the NUC algorithm 300 
substantially corrects both gain and bias non-uniformities independently of the calibration 
parameters. For simplification purposes, the description of NUC algorithm 300 is given for a 
15 two-dimensional sub-pixel motion. As would be understood by one having ordinary skill in the 
relevant art, the NUC algorithm 300 is not limited to the two-dimensional sub-pixel motion. 

Therefore, assuming two-dimensional sub-pixel motion, i.e., 0 < a < 1 and 0 < {$ < 1 , 
where a and p represent vertical and horizontal directional frame shifts between detectors' 
responses to scene irradiance, and that each detector has an arbitrary gain value, the following 
20 equation represents response by an (i, j) th detector to an irradiance z k (i, j) at time k, having a 
gain, a(i, j), and a bias, b(i, j): 

y k (/, ;) = a(i f j)z k (j* j) + b(U j) (15) 
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As described in equations (9)-(14), the bilinear interpolation model for the (i, j)* detector 
at time k+1, the detector model is now given by 

yM(hj) = a(iJ)z k+i (i,j) + b(i,j) (16) 
where 

5 z M (j, ;) = y x z k (i - 1, j - 1) + y 2 z k (i - 1, ;) + y 3 z k (i, + y 4 z k (i, j) (17) 

In this embodiment, the gain term for each of the detectors is expressed as follows: 
a(i,j) = l + Aa(iJ) (18) 
By substituting equations (16)-(18) into equation (11) above, the bias differential for a 
detector having an arbitrary two-dimensional shift is computed as follows: 
WJ) = 

1Q ft Aa(i - 1, j - \)z k (i - 1, j - 1) + y 2 Aa(/ - 1, j)z k (i - 1, j) + y^aji, j - l)z t (i, j - 1) (19) 

(i-r 4 ) 
a-r 4 ) 

Therefore, using bias differential in equation (19), the resulting bias estimate is as 
follows: 

b' (/, ;) = A(i, ;) + ~^—b r (i -1J-D + ~b r (j - 1, j) + ~-^—b r (i, j - 1) (20) 

(i-r 4 ) (i-r 4 ) 0-r 4 ) 

where each b r (i, j) term is a bias that has been updated during the previous correction 
15 cycle. From equation (20), the NUC algorithm 300 recursively computes gain and bias terms for 
each detector in the focal plane array. In an embodiment, assuming that initial values for gain 
and bias are a(i, j) = 1 and b(i, j) = 0 for all outer perimeter detectors 112, then Aa(i, j) = 0 and 
Hi, j) = 0. Thus, the bias differential for the top-leftmost non-calibrated detector is: 

A(2,2) = -Aa(2,2)z k (2,2) - b(2,2) (21) 
20 Then, 
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A'(2,2) = A(2,2) (22) 
Thus, in general, the bias estimate for the (i, j) 111 detector is: 

b r (i, j) = -Afl(i, ;)z* (f • 7) - ;) (23) 
Using the above equations, the multiple bias estimates for each detector are generated for 
5 various distinct image pairs. To reduce the effect of error in the bias estimate for each (i, j )* 
detector, the bias estimates are averaged over time. The bias estimates could also be combined 
using other conventional methods as would be clear to one with suitable skills in the relevant art. 
The expected value of the irradiance observed by an (i, j) th detector is defined as follows: 

MzdJ) = -£zJiJ) ( 24 ) 

10 wherein p is a total number of distinct image pairs. 

Then, the expected value of the quantity in equation (23) is: 

E{b r {U j) } = £{-Aa(i, j)z k (i, j) -*(/, j) } 

= -E{ Aa(i, j)z k (i, j)}- E{b(i, j)} 
= -Aa(i, j)E{ z k (/, ;) } - b(i, j) 
= -Aa(iJ)M z (iJ)-b(iJ) 

Thus, the bias estimate in the presence of gain non-uniformity is the true bias value plus 
the deviation of the gain from one multiplied by the average irradiance observed by the (i, j) 1 * 1 
15 detector. Therefore, the following equation expresses the correction for an (i, j) th detector's 
output by applying the above correction to equation (15): 



y k (i, j) + E{b r (/, j) } = z k (x, j) + tia(U j)z k (h j) + b(i, j) - ka(i y j)fi z (f , ;) - b(i, j) 
= z k (/, j) + ka(U j)[z k (i, ;) - M z (*'. J)\ 

Then, if true irradiance for each detector in the focal plane array is an average irradiance 

20 for an (i, j) th detector plus a deviation from this average irradiance, expressed as: 
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(27) 



then, equation (26) becomes: 



y k (/, ;) + E{b r (i, 7) } = z, (/, ;) + Aa(/, j)te k (i, ;) 



(28) 



Thus, the NUC algorithm substantially corrects gain non-uniformities when the observed 



5 irradiance at time k is sufficiently close to an average irradiance value (i.e., in this case, 

Az k (i, j) ~ 0 ). Also, if a detector's gain is approximately equal to one, the Aa term in the above 

equations is equal to zero, thus, the corrected detector gain value is approximately equal to the 
true irradiance value. This embodiment has been already described above with reference to gain 
term being uniform. 

10 The following description shows application of the NUC algorithm 300 to systems 

having temperature dependent bias as well as systems having non-uniform gain (a n ,s(ij) i- !)• 



15 equation (1) is non-uniform (i.e., a n ,s(i, j) 4- 1). In an alternate embodiment, the NUC algorithm 
can be generalized to correct for non-uniform gain. The NUC algorithm 300 is generalized to 
two pairs of readout signals at times m and n, y m , y m+ i and y n , y n +i exhibiting vertical shifts, <x m 
and <x„, respectively. If a(i, j) is the gain of the (i, j)* detector, then a ratio of a(i, j)/a(i+l, j) is 
computed from the detectors' readout signals y n (i, j), y n (i+l, j), y m (i, j), and y m (i+l, j), as 

20 follows: 



4. Correction for Non-uniform Gain 



As mentioned above, the present invention is capable of operating when the gain term in 




(29) 



aii + 1, j) a n [y m+x (i + 1, j) - [1 - a m )y n (i + 1, ;)] - a m [y n+l (i + 1, j) - [1 - a n ]y m (i + 1, j)] 
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Therefore, when the above quantity is multiplied by the value of (i+1, j)* detector of any 
raw time frame, the gain associated with the detector is effectively converted from a(i+l, j) to 
a(i, j). By applying this formula to each detector in FPA 106, each detector's gain becomes 
uniform. Then, the NUC algorithm 300 applies bias correction to detectors 115 of FPA 106 in 
5 the manner described above. 

FIGS. 5a-c illustrate application of NUC algorithm 300 to generate radiometrically 
accurate images. FIG. 5a illustrates a generated raw infrared (IR) image without application of 
NUC algorithm 300. FIG. 5b illustrates same IR imagery, as in FIG. 5a, corrected using a 
conventional two point calibration technique. FIG. 5c illustrates a radiometrically accurate 
10 image applying NUC algorithm 300, according to an embodiment of the present invention. The 
accuracy of the present invention's NUC algorithm is within 0.5% of the conventional technique. 



5. Optical System. 

FIG. 4 illustrates optical system 400 where the NUC algorithm 300 can be implemented 

15 to generate corrected image sequence collected by FPA 106, according to an embodiment of the 

present invention. Optical system 400 includes a first off-axis parabolic mirror 401, a movable 

field stop ("MFS") 450, a second off-axis parabolic mirror 402, and a camera apparatus 410 that 

includes FPA 106. MFS 450 is placed along an optical path between the first off-axis parabolic 

mirror 401 and the second off-axis parabolic mirror 402. The camera apparatus is placed along 

20 an optical path of the light reflected from the second off-axis parabolic mirror 402. The camera 

apparatus 410 further includes FPA 106 (as described in FIG. 1 above). 

The MFS 450 translates from position A to position B perpendicular an optical axis (not 

shown) between first off-axis parabolic mirror 401 and second off-axis parabolic mirror 402. In 

an alternate embodiment, MFS 450 is stationary and placed in the optical path between the first 
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off-axis parabolic mirror 401 and the second off-axis parabolic mirror 402. It acts as an optical 

switch re-directing the field of view of the outer perimeter detectors 1 12 (not shown). 

Scene light 407 is incident on the first off-axis parabolic mirror 401. The first off-axis 

parabolic mirror 401 reflects scene light 407 as a reflected scene light 411. The scene light 407 

5 includes four light ray bundles 407a, 407b, 407c, and 407d. Within each light ray bundle 407(a, 

b, c, d), each ray is traveling parallel to each other. 

The reflected scene light 41 1 is reflected in a different direction as compared to scene 

light 407, because of the properties of the off-axis parabolic mirror 401. Each light ray bundle 

407a, 407b, 407c, and 407d of scene light 407 is reflected by the first off-axis parabolic mirror 

10 401 as respective reflected light ray bundles 41 la, 41 lb, 41 lc, and 41 Id of reflected scene light 

411. These bundles are focused at the four points of the MFS 450. MFS 450 passes reflected 

scene light 411 at some points on MFS 450 and serves as a blackbody for reflected scene light 

41 1 at other points on MFS 450. 

The reflected scene light 411 then passes through the MFS 450. In an embodiment, 

15 reflected scene light ray bundles 41 lb and 41 lc are able to pass through the interior portion 412 

of MFS 450. However, MFS 450 does not allow reflected light ray bundles 41 la and 41 Id to 

pass through its outer portion 414 when MFS 450 is in position A. MFS 450 position A 

corresponds to the optical switch 120 "on" position 121 (not shown). On the other hand, MFS 

450 position B corresponds to the optical switch 120 "off position 122 (not shown). When MFS 

20 405 is in position B, all reflected light ray bundles 41 l(a, b, c, d) are able to pass through 

towards second off-axis mirror 402. In an embodiment, MFS 450 is a temperature controlled 

black body and acts as the calibration source 102 (not shown). 

After passing through MFS 450, reflected scene light 41 1 becomes image light 415 

having corresponding light ray bundles 415a, 415b, 415c, and 415d. The image light 415 travels 
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towards the second off-axis parabolic mirror 402. The second off-axis parabolic mirror 402 then 
reflects the image light 415 towards the camera apparatus 410 in a form of reflected image light 
419. The reflected image light 419 includes four light ray bundles 419a, 419b, 419c, and 419d 
corresponding to the original scene light ray bundles 407a, 407b, 407c, and 407d. Similarly to 
5 scene light 407, reflected image light ray bundles 419(a, b, c, d) are parallel to each other. The 
reflected image light 419 is then directed towards camera apparatus 410. The camera apparatus 
410 will generate a corrected image sequence. 

The camera apparatus 410 includes a receiving aperture 406 and an imaging apparatus 
405. The imaging apparatus 405 re-images the reflected light 419 onto the FPA 106 and corrects 

10 for any optical distortion introduced by the off-axis parabolic mirrors 401 and 402. The image 
generated by the camera apparatus 410 is substantially identical to the original scene for all 
detectors 1 15, when the optical switch 120 is in the "off position (or during the operation of the 
scene cycle of the NUC algorithm). When the optical switch 120 is in the "on" position (or 
during operation of the calibration cycle ) not all detectors 115 are viewing the scene. In 

15 particular, the outer perimeter detectors 112 (not shown) have their field of view directed 

towards the calibration source 102 and the inner detectors 116 (not shown) and buffer detectors 
114 (not shown) have their field of view directed to substantially identical image of the original 
scene. As can be understood by one having ordinary skill in the relevant art, other embodiments 
of the camera apparatus 410 are possible. 

20 

6. Conclusion. 

Example embodiments of the methods, circuits, and components of the present invention 

have been described herein. As noted elsewhere, these example embodiments have been 

described for illustrative purposes only, and are not limiting. Other embodiments are possible 
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and are covered by the invention. Such embodiments will be apparent to persons skilled in the 
relevant art(s) based on the teachings contained herein. Thus, the breadth and scope of the 
present invention should not be limited by any of the above-described exemplary embodiments, 
but should be defined only in accordance with the following claims and their equivalents. 
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